Research article about how to prototype documentary stories for VR

TekLab researchers Joakim Vindenes and Lars Nyre recently published a peer-reviewed article drawing on student active explorations of how to make documentary stories in VR.
Students in Media City Bergen demonstrated their VR journalism in June 2018. Stine Olsen Helland helps one of the guests. Photo: UiB.
Publisert: 22. mars 2024

TekLab has promoted student-active learning for more than a decade, and our insights are slowly being disseminated into the research community.

The title of the new research article is «Prototyping first-person viewer positions for VR narratives with storyboards and pilot productions». It was published in the Journal of Screenwriting issue number 3, 2023.

This journal is published by Intellect and is a leading research outlet in the field of screenwriting for television and film. The new article is part of a special issue ‘Virtual Reality: Exploring Technologies, Practices and Paradigms’. This issue explores the technologies and practices that VR storytelling implements, and pays particular attention to repurposing practices from earlier media, and specifically, screenwriting for film.

 

Why should we prototype VR storytelling?

The article deals with ways that creative teams can prototype documentary VR stories before making full-fledged and costly productions. Two prototyping methods are discussed; storyboarding that incorportates the 360 degree camera perspective of VR and short pilot productions that also explore possible narrative structures and qualities.

These two prototyping methods were adapted from interaction design and explored over two years by 40 bachelor students at the University of Bergen. You can read about the results of these student project in an article at Vismedia "Exploring VR journalism in higher education" (2018).

These educational projects were backed by the Vismedia project funded by the Norwegian Research Coundil (NFR) and TekLab funded by the Norwegian Directorate of Higher Education and Competence (HK-dir).

 

Here we see a three-dimensional and dynamic approach to VR-storyboarding. The creative team can move the various parts of the scene around, to explore alternatives. The first-person perspective is represented as the camera, and the actors and items can be moved dynamically to plan for vari- ous scenes. Post-it notes were used to keep track of various items in the scene, and arrows were used to indicate the movement of the actors with moveable dotted lines indicating the field of view.

 

Exploring first-person perspectives

Here we see a third-person point of view of a kitchen. In the middle of the illustration, a 360-degree camera tripod is placed in the scene, representing the position where the user or camera should be situated.

When making narratives in VR the story world has to be experienced from a first-person perspective. And the storytelling has to take place in relation to the user's ego-centric and immersive perspective. The students explored ways of planning and testing stories that have this novel perspective on a story world. Joakim Vindenes explains:

– We provide analyses of how storyboards and pilot productions can situate the user in relation to the virtual environment, and also developed a vocabulary to describe them. The students were asked to first create storyboards and then short pilot versions of their VR stories. They produced the pilot stories for Samsung Gear and HTC Vive, says Joakim.

– Based on the students’ trials, the article evaluates the use value of these methods for constructing first-person viewer positions for VR narratives. Traditionally, producers would use screenwriting methods, and we are happy that the Journal of Screenwriting, and its reviewers, acknowledged our HCI-driven approach.

Here we see a circular bird’s-eye storyboard with a specified field of view. It was later revised so that the field of view could be changed dynamically. This was accomplished by pinning a new paper circle with a 90-degree cut-out on top of the storyboard, which could then be rotated to explore the various points of view of the given environment.

 

Student led research

The main author is PhD Joakim Vindenes who is a Senior Analyst IT / XR developer at Equinor. The second author is professor Lars Nyre who is a teacher and researcher in Media City Bergen.

While conducting his PhD research, Joakim was also a co-teacher of virtual reality at the bachelor in media and interaction design (MIX). He was instrumental in the student explorations that led to the empirical material for the Journal of Screenwriting-article.

Joakim has been involved with interaction design and VR since 2012 when he started at the bachelor program in what was then called "new media". He completed his master's thesis in information science in 2017 and then embarked on a PhD project at Slate.  Joakim was loosely affiliated also with Vismedia during the period when he worked on his doctoral project. All his research is based on combining HCI with VR. Joakim completed his PhD in 2023.

The second author Lars Nyre was also involved in the courses, along with Fahmy Zulfikar and Audun Klyve Gulbrandsen at Media City Bergen.


Publisert: 22. mars 2024
linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram